Combining ConvNets with Hand-Crafted Features for Action Recognition Based on an HMM-SVM Classifier

摘要

This paper proposes a new framework for RGB-D-based action recognition thattakes advantages of hand-designed features from skeleton data and deeplylearned features from depth maps, and exploits effectively both the local andglobal temporal information. Specifically, depth and skeleton data are firstlyaugmented for deep learning and making the recognition insensitive to viewvariance. Secondly, depth sequences are segmented using the hand-craftedfeatures based on skeleton joints motion histogram to exploit the localtemporal information. All training se gments are clustered using an InfiniteGaussian Mixture Model (IGMM) through Bayesian estimation and labelled fortraining Convolutional Neural Networks (ConvNets) on the depth maps. Thus, adepth sequence can be reliably encoded into a sequence of segment labels.Finally, the sequence of labels is fed into a joint Hidden Markov Model andSupport Vector Machine (HMM-SVM) classifier to explore the global temporalinformation for final recognition.